Subpixel rendering filters for high brightness subpixel layouts

ABSTRACT

A display system comprises a display panel substantially comprising a subpixel repeating group tiled across the panel in a regular pattern. The subpixel repeating group comprises at least one white subpixel and a plurality of colored subpixels. The display system further comprises input circuitry configured to receive input image data indicating an image for rendering on the display panel, and subpixel rendering circuitry configured to compute an output luminance value for each subpixel of the display panel. The subpixel rendering circuitry multiplies data values of a spatial portion of the input image data by at least one image filter kernel which comprises a matrix of coefficients arranged such that each coefficient represents a fractional part of one of said data values of the spatial portion of the input image data. The subpixel rendering circuitry is further configured to sharpen the output luminance values using a luminance signal.

RELATED APPLICATIONS

This application is a divisional of, and claims priority to, U.S. patentapplication Ser. No. 11/780,898 filed on Jul. 7, 2007 which is acontinuation of, and claims priority to, U.S. patent application Ser.No. 10/821,388 filed on Apr. 9, 2004, and issued as U.S. Pat. No.7,248,268.

BACKGROUND

In commonly owned United States Patent Applications: (1) U.S. patentapplication Ser. No. 09/916,232, entitled “ARRANGEMENT OF COLOR PIXELSFOR FULL COLOR IMAGING DEVICES WITH SIMPLIFIED ADDRESSING,” filed Jul.25, 2001, now issued as U.S. Pat. No. 6,903,754 (“the '754 patent”); (2)U.S. patent application Ser. No. 10/278,353, entitled “IMPROVEMENTS TOCOLOR FLAT PANEL DISPLAY SUB-PIXEL ARRANGEMENTS AND LAYOUTS FORSUB-PIXEL RENDERING WITH INCREASED MODULATION TRANSFER FUNCTIONRESPONSE,” filed Oct. 22, 2002, and published as US Patent PublicationNo. 2003/0128225 (“the '225 application”); (3) U.S. patent applicationSer. No. 10/278,352, entitled “IMPROVEMENTS TO COLOR FLAT PANEL DISPLAYSUB-PIXEL ARRANGEMENTS AND LAYOUTS FOR SUB-PIXEL RENDERING WITH SPLITBLUE SUB-PIXELS,” filed Oct. 22, 2002 now issued as U.S. Pat. No.7,417,648 (“the '648 patent”); (4) U.S. patent application Ser. No.10/243,094, entitled “IMPROVED FOUR COLOR ARRANGEMENTS AND EMITTERS FORSUB-PIXEL RENDERING,” filed Sep. 13, 2002 and published as US PatentPublication No. 2004/0051724 (“the '724 application”); (5) U.S. patentapplication Ser. No. 10/278,328, entitled “IMPROVEMENTS TO COLOR FLATPANEL DISPLAY SUB-PIXEL ARRANGEMENTS AND LAYOUTS WITH REDUCED BLUELUMINANCE WELL VISIBILITY,” filed Oct. 22, 2002 and published as USPatent Publication No. 2003/0117423 (“the '423 application”); (6) U.S.patent application Ser. No. 10/278,393, entitled “COLOR DISPLAY HAVINGHORIZONTAL SUB-PIXEL ARRANGEMENTS AND LAYOUTS,” filed Oct. 22, 2002 nowissued as U.S. Pat. No. 7,283,142 (“the '142 patent”); (7) U.S. patentapplication Ser. No. 10/347,001 entitled “IMPROVED SUB-PIXELARRANGEMENTS FOR STRIPED DISPLAYS AND METHODS AND SYSTEMS FOR SUB-PIXELRENDERING SAME,” filed Jan. 16, 2003, and published as US PatentPublication No. 2004/0080479 (“the '479 application”); each of which isherein incorporated by reference in its entirety, novel sub-pixelarrangements are disclosed for improving the cost/performance curves forimage display devices.

For certain subpixel repeating groups having an even number of subpixelsin a horizontal direction, the following systems and techniques toaffect improvements, e.g. proper dot inversion schemes and otherimprovements, are disclosed and are herein incorporated by reference intheir entirety: (1) U.S. patent application Ser. No. 10/456,839 entitled“IMAGE DEGRADATION CORRECTION IN NOVEL LIQUID CRYSTAL DISPLAYS” andpublished as US Patent Publication No. 2004/0246280 (“the '280application”); (2) U.S. patent application Ser. No. 10/455,925 entitled“DISPLAY PANEL HAVING CROSSOVER CONNECTIONS EFFECTING DOT INVERSION” andpublished as US Patent Publication No. 2004/0246213 (“the '213application”); (3) U.S. patent application Ser. No. 10/455,931 entitled“SYSTEM AND METHOD OF PERFORMING DOT INVERSION WITH STANDARD DRIVERS ANDBACKPLANE ON NOVEL DISPLAY PANEL LAYOUTS” now issued as U.S. Pat. No.7,218,301 (“the '301 patent”); (4) U.S. patent application Ser.No.10/455,927 entitled “SYSTEM AND METHOD FOR COMPENSATING FOR VISUALEFFECTS UPON PANELS HAVING FIXED PATTERN NOISE WITH REDUCED QUANTIZATIONERROR” now issued as U.S. Pat. No. 7,209,105 (“the '105 patent”); (5)U.S. patent application Ser. No. 10/456,806 entitled “DOT INVERSION ONNOVEL DISPLAY PANEL LAYOUTS WITH EXTRA DRIVERS” now issued as U.S. Pat.No. 7,187,353 (“the '353 patent”); (6) U.S. patent application Ser. No.10/456,838 entitled “LIQUID CRYSTAL DISPLAY BACKPLANE LAYOUTS ANDADDRESSING FOR NON-STANDARD SUBPIXEL ARRANGEMENTS” now issued as U.S.Pat. No. 7,397,455 (“the '455 patent”); (7) U.S. patent application Ser.No. 10/696,236 entitled “IMAGE DEGRADATION CORRECTION IN NOVEL LIQUIDCRYSTAL DISPLAYS WITH SPLIT BLUE SUBPIXELS”, filed Oct. 28, 2003 andpublished as US Patent Publication No. 2005/0083277 (“the '277application”); and (8) U.S. patent application Ser. No. 10/807,604entitled “IMPROVED TRANSISTOR BACKPLANES FOR LIQUID CRYSTAL DISPLAYSCOMPRISING DIFFERENT SIZED SUBPIXELS”, filed Mar. 23, 2004 now issued asU.S. Pat. No. 7,268,758 (“the '758 patent”);.

These improvements are particularly pronounced when coupled withsub-pixel rendering (SPR) systems and methods further disclosed in thoseapplications and in commonly owned United States Patent Applications:(1) U.S. patent application Ser. No. 10/051,612, entitled “CONVERSION OFA SUBPIXEL FORMAT DATA TO ANOTHER SUB-PIXEL DATA FORMAT,” filed Jan. 16,2002 now issued as U.S. Pat. No. 7,123,277 (“the '277 patent”); (2) U.S.patent application Ser. No. 10/150,355, entitled “METHODS AND SYSTEMSFOR SUB-PIXEL RENDERING WITH GAMMA ADJUSTMENT,” filed May 17, 2002 nowissued as U.S. Pat. No. 7,221,381 (“the '381 patent”); (3) U.S. patentapplication Ser. No. 10/215,843, entitled “METHODS AND SYSTEMS FORSUB-PIXEL RENDERING WITH ADAPTIVE FILTERING,” filed Aug. 8, 2002 nowissued as U.S. Pat. No. 7,184,066 (“the '066 patent”); (4) U.S. patentapplication Ser. No. 10/379,767 entitled “SYSTEMS AND METHODS FORTEMPORAL SUB-PIXEL RENDERING OF IMAGE DATA” filed Mar. 4, 2003 andpublished as U.S. Patent Publication No. 2004/0196302 (“the '302application”); (5) U.S. patent application Ser. No. 10/379,765 entitled“SYSTEMS AND METHODS FOR MOTION ADAPTIVE FILTERING,” filed Mar. 4, 2003now issued as U.S. Pat. No. 7,167,186 (“the '186 patent”); (6) U.S.patent application Ser. No. 10/379,766 entitled “SUB-PIXEL RENDERINGSYSTEM AND METHOD FOR IMPROVED DISPLAY VIEWING ANGLES” filed Mar. 4,2003 now issued as U.S. Pat. No. 6,917,368 (“the '368 patent”); (7) U.S.patent application Ser. No. 10/409,413 entitled “IMAGE DATA SET WITHEMBEDDED PRE-SUBPIXEL RENDERED IMAGE” filed Apr. 7, 2003, now issued asU.S. Pat. No. 7,352,374 (“the '374 patent”); which are herebyincorporated herein by reference in their entirety.

Improvements in gamut conversion and mapping are disclosed in commonlyowned and co-pending United States Patent Applications: (1) U.S. patentapplication Ser. No. 10/691,200 entitled “HUE ANGLE CALCULATION SYSTEMAND METHODS”, filed Oct. 21, 2003 now issued as U.S. Pat. No. 6,980,219(“the '219 patent”); (2) U.S. patent application Ser. No. 10/691,377entitled “METHOD AND APPARATUS FOR CONVERTING FROM SOURCE COLOR SPACE TORGBW TARGET COLOR SPACE”, filed Oct. 21, 2003 and published as US PatentPublication No. 2005/0083341 (“the '341 application”); (3) U.S. patentapplication Ser. No. 10/691,396 entitled “METHOD AND APPARATUS FORCONVERTING FROM A SOURCE COLOR SPACE TO A TARGET COLOR SPACE”, filedOct. 21, 2003 and published as US Patent Publication No. 2005/0083352(“the '352 application”) and (4) U.S. patent application Ser. No.10/690,716 entitled “GAMUT CONVERSION SYSTEM AND METHODS” filed Oct. 21,2003 now issued as U.S. Pat. No. 7,176,935 (“the '935 patent”); whichare all hereby incorporated herein by reference in their entirety.

Additional advantages have been described in (1) U.S. patent applicationSer. No. 10/696,235 entitled “DISPLAY SYSTEM HAVING IMPROVED MULTIPLEMODES FOR DISPLAYING IMAGE DATA FROM MULTIPLE INPUT SOURCE FORMATS”,filed Oct. 28, 2003, now issued as U.S. Pat. No. 7,084,923 (“the '923patent”); and (2) U.S. patent application Ser. No. 10/696,026 entitled“SYSTEM AND METHOD FOR PERFORMING IMAGE RECONSTRUCTION AND SUBPIXELRENDERING TO EFFECT SCALING FOR MULTI-MODE DISPLAY” filed Oct. 28, 2003and published as US Patent Publication No. 2005/0088385 (“the '385application”).

Additionally, these co-owned and co-pending applications are hereinincorporated by reference in their entirety: (1) U.S. patent applicationSer. No. 10/821,387 entitled “SYSTEM AND METHOD FOR IMPROVING SUB-PIXELRENDERING OF IMAGE DATA IN NON-STRIPED DISPLAY SYSTEMS” and published asUS Patent Publication No. 2005/0225548 (“the '548 application”); (2)U.S. patent application Ser. No. 10/821,386 entitled “SYSTEMS ANDMETHODS FOR SELECTING A WHITE POINT FOR IMAGE DISPLAYS” now issued asU.S. Pat. No. 7,301,543 (“the '543 patent”); (3) U.S. patent applicationSer. No. 10/821,353 entitled “NOVEL SUBPIXEL LAYOUTS AND ARRANGEMENTSFOR HIGH BRIGHTNESS DISPLAYS” and published as US Patent Publication No.2005/0225574 (“the '574 application”); (4) U.S. patent application Ser.No. 10/821,306 entitled “SYSTEMS AND METHODS FOR IMPROVED GAMUT MAPPINGFROM ONE IMAGE DATA SET TO ANOTHER” and published as US PatentPublication No. 2005/0225562 (“the '562 application”); which are allhereby incorporated by reference. All patent applications mentioned inthis specification are hereby incorporated by reference in theirentirety.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in, and constitute apart of this specification illustrate exemplary implementations andembodiments of the invention and, together with the description, serveto explain principles of the invention.

FIGS. 1 through 3B are embodiments of high brightness layouts fordisplays of all types as made in accordance with the principles of thepresent invention.

FIG. 4 is one exemplary embodiment of a resampling of one of the colorplanes for one of the above high brightness layouts.

FIGS. 5A and 5B are yet other embodiments of a high brightness layoutfor displays as made in accordance with the principles of the presentinvention.

FIG. 6 is one exemplary embodiment of a resampling of one of the colorplanes for the layout as shown in FIG. 5.

FIGS. 7 and 8 are yet other embodiments of high brightness layouts fordisplays as made in accordance with the principles of the presentinvention.

FIG. 9 is one exemplary embodiment of a resampling of one of the colorplanes for the layout as shown in FIG. 8.

FIG. 10 is one example of a reconstruction grid being superimposed ontoa target 3-color subpixel layout.

FIGS. 11 through 14C are examples of various resample areas depending onthe relative positioning of input image data grid to target subpixellayout.

FIG. 15 is another embodiment of the relative position of a 3-colortarget subpixel layout shifted with respect to an input image data grid.

FIG. 16A through 18C are examples of various resample areas for theexample of FIG. 15.

DETAILED DESCRIPTION

Reference will now be made in detail to implementations and embodiments,examples of which are illustrated in the accompanying drawings. Whereverpossible, the same reference numbers will be used throughout thedrawings to refer to the same or like parts.

Subpixel Rendering for Five Color Systems w/White

FIG. 1 shows one embodiment of a portion of a high-brightness,multiprimary display 100 substantially comprising a subpixel repeatinggroup 102 as shown. Group 102 is an octal subpixel repeating groupcomprising white (or no color filter) subpixels 104, red subpixels 106,green subpixel 108, blue subpixels 110 and cyan subpixels 112. The whitesubpixel is added to help achieve the high brightness performance of thedisplay. Additionally, as the white subpixels are good candidates forbeing centers of luminance for subpixel rendering (SPR)—the white, asthe majority subpixel, gives high MTF Limit performance. In thisembodiment, there are equal numbers of red, green, cyan, and bluesubpixels—of course, other embodiments may deviate some from this colorpartitioning. Given that the white subpixel is adding brightness to thesystem and that the use of the cyan color is to give a wider colorgamut, it may be advantageous to set the color points of the minoritysubpixels to be deeply saturated to result in a wide color gamut. Itshould be noted that these color points and energies are only“substantially” the colors described as “red”, “green”, “blue”, “cyan”,and “white”. The exact color points may be adjusted to allow for adesired white point when all of the subpixels are at their brighteststate.

FIG. 2 shows a portion of another embodiment of a high brightness,5-color display. Here, the subpixel repeating group is group 202—whichis larger than the one shown in FIG. 1 because the color subpixels areplaced on a hexagonal grid. One possible advantage of a hexagonal gridis that it tends to scatter the Fourier energies in more directions andpoints. This may be especially useful for the dark luminance wellscaused by the blue subpixels. Another possible advantage is that eachrow contains all four colors as well as the white subpixels, allowingfor horizontal lines to be black and white, fully sharpened, withoutchromatic aliasing.

One possible embodiment of a display system using this layout mayprocess image data and render it as follows:

-   (1) Convert conventional data (e.g. RGB, sRGB, YCbCr, or the like)    to RGBCW+L image data, if needed;-   (2) Subpixel render each individual color plane;-   (3) Use the “L” (or “Luminance”) plane to sharpen each color plane.

The subpixel rendering filter kernels may be constructed from arearesampling theory, as disclosed earlier in many incorporatedapplications noted above. Both layouts may be subpixel rendered fromdata sets that have a one-to-one mapping. That is to say, one incomingconventional pixel maps to one white subpixel. The white subpixels maythen fully reconstruct the bulk of the non-saturated luminance signal ofthe image. The surrounding colored subpixels then operate to provide thecolor signal. The incoming image may be any format of color signal, aslong as color gamut mapping with or without gamut expansion, may operateto convert said format to RGBCW expected by the subpixel renderingengine. It will be appreciated that such area resampling filters may bereplaced by other suitable subpixel rendering techniques: resamplingusing bicubic filter, sinc filters, windowed-sinc filter and anyconvolutions thereof. It will be further appreciated that the scope ofthe present invention encompasses the use of these other techniques.

As the white subpixels are mapped one to one, they may use a unityfilter with no further processing required. The color planes may befiltered using several possible kernels. For example, assuming that theimage is band-limited, one embodiment might shift the phase of each ofthe color planes and the Luminance plane to the interstitial positionsof the color subpixels in the horizontal direction. This may beaccomplished with a simple cubic interpolation filter: −1/16, 9/16,9/16, −1/16. It should be note that the white plane may not need to beshifted. For non-band-limited images (e.g. text or sharp edges inimages), there may not need to be the cubic filtered phase shift asabove.

Then, the color planes may be filtered with an area resample filter. ADifference of Gaussian (DOG) filter applied to luminance may optionallybe added, examples are given here:

$\begin{matrix}1 & 2 & 1 & \; \\2 & 4 & 2 & \; \\1 & 2 & 1 & \left( {{Divide}\mspace{14mu}{by}\mspace{14mu} 16} \right)\end{matrix}$

-   -   Area Resample Filter for hexagonal and square arrangement

$\begin{matrix}0 & 0 & {- 2} & 0 & 0 & \; \\{- 1} & 0 & 0 & 0 & {- 1} & \; \\0 & 0 & 8 & 0 & 0 & \; \\{- 1} & 0 & 0 & 0 & {- 1} & \; \\0 & 0 & {- 2} & 0 & 0 & \left( {{Divide}\mspace{14mu}{by}\mspace{14mu} 16} \right)\end{matrix}$

-   -   DOG Filter for hexagonal arrangement of FIG. 2.

It should be noted that non-zero values coincide with the same color tokeep the color balanced. Using the luminance signal implements asimplified “cross-color” sharpening.

In another embodiment, one could also perform actual cross-colorsharpening, distributing the values of the cross-color coefficientsamong the color filter kernels such that the matrices add up to thedesired numbers such as above. One method that may be useful is todivide the values of the actual subpixel luminances—red, green, blue,and cyan—by the luminance value of the color that is being sharpened,and then multiply the result by the matrix above times a suitablenormalization constant such that it adds up to the matrix above. Anotherway might be to not perform the normalization, which would mean thatsome colors would experience greater than unity gain sharpening. Thecolors that experienced the greatest gain would be the colors with thelowest luminance. This last property may be useful to reduce the“dottiness” of the high spatial frequency detail, increasing the qualityof the signal. These methods and techniques of using varying sharpeninggain on the colors may also be driven by the luminance signal as above.

In one embodiment, multiplying the values of the sharpening matrix by aconstant allows adjustment of the gain of the system. For thisembodiment, if the constant is less than one, the filter is softer; ifthe constant is greater than one, the filter is sharper. Of course,other embodiments are contemplated by the present invention withdifferent matrices and constants.

It should also be noted that the one possible method uses the simplestsubpixel rendering filter kernels—with the math being performedsubstantially by bit shift division and addition. Other methods andembodiments may give numbers that require more multi-bit precisionmultipliers. Of course, performing the color gamut mapping may requiresuch multipliers as well.

As well as cross-color sharpening, one embodiment of the system may beimplemented using self-sharpening by adding the two matrices together.For example, the following may be useful for the arrangement of FIG. 2:

$\begin{matrix}0 & 0 & {- 2} & 0 & 0 & \; \\{- 1} & 1 & 2 & 1 & {- 1} & \; \\0 & 2 & 12 & 2 & 0 & \; \\{- 1} & 1 & 2 & 1 & {- 1} & \; \\0 & 0 & {- 2} & 0 & 0 & {{Divide}\mspace{14mu}{by}\mspace{14mu} 16}\end{matrix}$

Since the mapping of the conventional pixel data, in what ever form itcomes in, to the multi-primary space is indeterminate, this mayintroduce a degree of freedom that could be advantageous. For example,choosing any given algorithm may always give the right color over all;but may not give the best visual result. For example, the colorsubpixels, not all having the same luminance, may introduce a spuriouspattern for many non-optimal mappings. The desired color mapping wouldgive the most even texture for patches of a given color, minimizingvisible spatial frequencies of luminance modulation, over the broadestrange of colors; hue, saturation, and brightness. Such a mapping wouldallow the fine details to be displayed using the algorithm disclosedabove. In another embodiment, the system might work with a plurality oftransform matrices, if no single transform matrix provides optimalresult for all colors. It may be advantageous to create domains, or evencontinuously variable transforms.

Rendering Novel RGBW Panels

In many cases, novel RGBW panels (and 5-, 6-, n-color panels, for thatmatter) will be called upon to render legacy RGB or other 3-color imagedata. In many applications incorporated by reference above, there aredescribed various embodiments for subpixel rendering resampling amodified conventional image data set. The modification is that each andevery incoming conventional pixel has four (or more)—instead ofthree—color component values; e.g. Red, Green, Blue, and “White”. The“White” in quotes denotes that this color point may or may not be at thewhite point of the display when all color subpixels are set to theirmaximum values. It may be desirable that any Gamut Mapping Algorithm(GMA) conversion from RGB to RGBW (or other multiprimary color space)occur before the subpixel rendering to keep the image from beingblurred. The filter set could be designed to produce good results forboth text and photographs. For example, in the '724 applicationincorporated by reference, there are shown some novel RGBW and RGBClayouts. For these layouts, one embodiment of the filters for the SPRfor layouts that have a red/green checkerboard may be the following:

Red and Green use:

$\begin{matrix}{- {.0625}} & 0 & {- {.0625}} & \; & 0 & {.125} & 0 & \; & {- {.0625}} & {.125} & {.0625} \\0 & {.25} & 0 & + & {.125} & {.5} & {.125} & = & {.125} & {.75} & {.125} \\{- {.0625}} & 0 & {- {.0625}} & \; & 0 & {.125} & 0 & \; & {- {.0625}} & {.125} & {.0625}\end{matrix}$DOG  Wavelet + Area  Resample = Cross-Color  Sharpening  Kernel

The Red and Green color planes are area resampled to remove any spatialfrequencies that will cause chromatic aliasing. The DOG wavelet is usedto sharpen the image using the cross-color component. That is to say,the red color plane is used to sharpen the green subpixel image and thegreen color plane is used to sharpen the red subpixel image. This allowsthe cross-color luminance signal to be impressed onto the colorsubpixels, ‘filling in the holes’ in color images. It should be notedthat for monochromatic images, the results of cross-color DOG waveletsharpening is the same as self-color sharpening. It should also be notedthat the coefficients disclosed above are exemplary of one particularembodiment and that the present invention contemplates many othermatrices having suitable coefficients that suffice.

The Blue color plane may be resampled using one of a plurality offilters. For example, blue could be resampled with a simple 2×2 boxfilter:

$\begin{matrix}{.25} & {.25} \\{.25} & {.25}\end{matrix}$

Alternatively, the Blue color plane could be resampled using a box-tentfilter centered on the blue subpixel:

$\begin{matrix}{.125} & {.25} & {.125} \\{.125} & {.25} & {.125}\end{matrix}$

Moreover, the white plane could also be filtered using one of aplurality of filters. For example, the white or cyan color plane couldbe resampled using a non-axis-separable 4×4 box-cubic filter:

$\begin{matrix}\; & {{- 1}/32} & {{- 1}/32} & \; \\{{- 1}/32} & {10/32} & {10/32} & {{- 1}/32} \\{{- 1}/32} & {10/32} & {10/32} & {{- 1}/32} \\\; & {{- 1}/32} & {{- 1}/32} & \;\end{matrix}$

Alternatively, to help abate that there is no phase error, nor aliasing,on the white or cyan subpixel, an axis-separable 3×4 tent-cubic filtermight be used:

$\begin{matrix}{{- 1}/64} & {{- 1}/32} & {{- 1}/64} \\{9/64} & {9/32} & {9/64} \\{9/64} & {9/32} & {9/64} \\{{- 1}/64} & {{- 1}/32} & {{- 1}/64}\end{matrix}$

The use of the box-cubic and tent-cubic filters may help to reduce themoirè artifacts in photographs while maintaining sharpness in text bytaking advantage of the mid-position of the white subpixels. Althoughnot necessary, it is possible to use the same filters for both blue andwhite color planes. One could use either the plain box or tent for both,or the box-cubic or tent-cubic for both. Alternatively, the cubicfilters should be chosen for both.

FIGS. 3A and 3B show embodiments of a high brightness display having therepeating subpixel groupings as shown. Although these layouts may haveany aspect ratio possible, FIGS. 3A and 3B depicts this layout with allsubpixels having a 1:3 aspect ratio. That produces subpixels that aretaller and thinner than a possible square outline or 2:3 aspect ratio.This layout comprises a combination where the blue sub-pixels have thesame size as the red and green and the same number —which results in asubstantially color-balanced RGBW layout, since there is the same areacoverage of the red, green, and blue emitters using the same filters aswould be found in conventional RGB display panels.

The layouts of FIGS. 3A and 3B have a potential advantage in that it maybe manufactured on a standard RGB stripe backplane with a suitablechange in the color filter. One embodiment of a panel having one ofthese layouts may use any suitable form of SPR algorithm, as discussedherein or in applications incorporated by reference.

In one embodiment, the image source data to the display might assume asquare aspect ratio—thus, with no scaling, each input pixel would map tothree sub-pixels in this layout. However, these RGBW 1:3 layouts are 4sub-pixels wide per repeat cell. If source pixels are mapped to groupsof three such sub-pixels, then three of the layouts tiled horizontallymight suffice before all the possible combinations are found. For eachdifferent combination of three output sub-pixels grouped like this, adifferent set of area resample filters might suffice. This is similar tothe process of finding a repeat cell and generating different sets offilters for scaling, as disclosed in applications incorporated above.

In fact, the same logic that does scaling might be used to selectsuitable filters. In one embodiment, there could be a simplificationthat may be easier to implement than scaling. As in scaling, there maybe symmetries that reduce the total number of filters, and in this case,there are only three filters that are used over and over again indifferent combinations of colors. FIG. 4 depicts the resample areas andfilters so generated for the red subpixels. The filters for green, blueand white are identical, but appear in a different order or orientation.

As may be seen in FIG. 4, the resample areas may be hexagons with threedifferent alignments: offset ⅓ to the left (as seen as areas 404),centered (as seen as areas 406), or offset ⅓ to the right (as seen asarea 402). The three resulting unique area resampling filters are:

Area Resample Filters 2 12 0 0 14 0 0 2 12 82 146 0 22 184 22 0 146 82 212 0 0 14 0 0 2 12

The resulting images may have a slightly blurred appearance, and thus,it may be possible to apply cross-luminosity sharpening filters tosubstantially correct this:

Cross Luminosity Filters −8 −8 0 −8 0 −8 0 −8 −8 0 32 0 0 32 0 0 32 0 −8−8 0 −8 0 −8 0 −8 −8

It will be appreciated that these cross-luminosity filters aredistinguishable from cross-color sharpening filters. One possibleadvantage of cross-luminosity filtering is that blue and white can besharpened, as well as red and green (as before with cross-color) with asingle value, thus reducing the number of operations. In a low cost RGBWimplementation, these luminosity values may be calculated using any ofthe embodiments disclosed in several applications incorporated herein.One example uses the formula:Y=(2*R+4*G+G+B)/8.

It should be noted that this luminosity value can be calculated byperforming only shifts and adds in hardware or software.

In one embodiment, the color values may be sampled using the arearesample filters above, the luminosity “plane” may be sampled using thecross-luminosity filters, and the two results are added together. Thiscan occasionally produce values below zero or above the maximum, so theresults may be clamped to the allowed range.

The area resampling filters above correct for the offset position of thesub-pixel within the source pixel with coefficients that sample a littlemore of the color to one side or the other. An alternative way toaccomplish this may be to use a horizontal cubic filter to change thephase of the input data. When an output sub-pixel lands in the center ofan input pixel, no phase adjustment is necessary and the centered arearesample filter can be used. When the output sub-pixel lands in anoffset position in an input pixel, one of the following two cubicfilters may be used to generate a “pseudo-sample” that is aligned withthe center of the output sub-pixel:

Horizontal Cubic Filters −9 84 199 −18 −18 199 84 −9

Once the phase is aligned, the centered area resample filter andsharpening filter may be used for all output sub-pixels. In oneexemplary hardware implementation, these cubic filters may beimplemented using special purpose logic to do the multiplies by fixednumbers. This calculation could be done on input values before passingthem to the sub-pixel rendering logic. The sub-pixel rendering logic maythus be simplified, at the cost of the pre-conditioning of the data withthe cubic filter. In one exemplary software implementation, it might beadvantageous to convolve the cubic filters with the centered arearesample filter. This results in two filter kernels shown below:

Cubic plus Area Resampling Filters 0 0 4 11 −1 0 0 −1 11 4 0 0 −1 1 77149 4 −2 −2 4 149 77 1 −1 0 0 4 11 −1 0 0 −1 11 4 0 0

These two filters can be substituted for the offset filters in the firstarea resampling case to simulate the cubic case with no other changes tothe software. When these filters are used, the luminosity plane may alsobe phase aligned which might employ convolving the centered sharpeningfilter with the two horizontal cubic filters:

Cubic plus Sharpening Filters 1 −6 −2 −6 −3 −3 −6 −2 −6 1 0 −2 25 11 −2−2 11 25 −2 0 1 −6 −2 −6 −3 −3 −6 −2 −6 1

As the layouts of FIGS. 3A and 3B are similar to the conventional RGBstripe layout, one low cost system might proceed by copying or assigningthe nearest RGB or W value into the output sub-pixel without performingarea resampling. However, undesirable color error might occur. Thehorizontal component of this error may be reduced by using thehorizontal cubic filters above. As this system would require no linebuffers, low hardware costs reduce the overall cost of the system.Additionally, as the cubic filters have a slight sharpening effect,separate sharpening may not be not needed. The horizontal lines of fontsmay look reasonably good, however the vertical components of fonts maystill exhibit color error. Such a low cost system might be acceptable inan image-only application, such as a camera viewfinder.

FIGS. 5A and 5B are yet other embodiments of a high brightness RGBWlayout—but have a 1:2 aspect ratio for their subpixels. This subpixelrepeating group comprising blue sub-pixels the same size as the red andgreen and adding two white subpixels tends to result in a color-balancedRGBW layout. It will be appreciated that the layouts of FIGS. 3A, 3B,5A, and 5B—while placing the red and green subpixels and the blue andwhite subpixels, or red and blue subpixels and the green and whitesubpixels, on a checkerboard pattern—may be viewed as having otherpatterns alternatively. For example, any mirror image or rotation orother symmetries are contemplated. Additionally, the subpixels need notbe placed on a fully intertwined checkerboard for the purposes of thepresent invention, an example of which is given in FIG. 7.

In one embodiment, each input pixel image data may be mapped to twosub-pixels. In effecting this, there are still a number of differentways to align the input pixels and generate the area resampling filters.The first considered was to simply align 4 input pixels directly withthe layouts shown in FIGS. 5A and 5B. FIG. 6 shows one example of anarea resampling of the red color plane as described. Input pixel imagedata is depicted on grid 602 and the repeating group 604 of subpixels ofFIG. 5A is superimposed upon the grid. Red subpixels 606 and 610 andtheir associated “diamond” filters 608 and 612 are also shown. Arearesampling may then occur in the manner described herein and in manyapplications incorporated herein, an example is given here:

$\begin{matrix}{- {.0625}} & 0 & {- {.0625}} & \; & 0 & {.125} & 0 & \; & {- {.0625}} & {.125} & {.0625} \\0 & {.25} & 0 & + & {.125} & {.5} & {.125} & = & {.125} & {.75} & {.125} \\{- {.0625}} & 0 & {- {.0625}} & \; & 0 & {.125} & 0 & \; & {- {.0625}} & {.125} & {.0625}\end{matrix}$DOG  Wavelet + Area  Resample = Luminance  Sharpening  Kernel

For non-band-limited images, such as text, computer aided drafting(CAD), line art, or other computer generated images, it may beadvantageous to treat pairs of subpixels as though they weresubstantially coincident, using the substantially exact same filterkernel to resample the image. This will result in sharp verticals andhorizontal lines being reconstructed.

Alternatively, these diamond filters may be offset by ¼ of an inputpixel. For a panel with the arrangement of FIG. 5A, the filter kernels,shown below, may be substantially the same for red and green; while blueand white use filters may be offset in the opposite directionhorizontally.

Red/green Blue/white 4 28 0 0 28 4 64 120 8 8 120 64 4 28 0 0 28 4

Another embodiment might offset the input pixels until their centerpoints are aligned with the centers of some of the repeating sub-pixels.One example of filters that may suffice are as follows:

Red/green (or blue/white) Blue/white (or red/green, respectively) 0 32 016 16 32 128 32 96 96 0 32 0 16 16

One of these is the “diamond” filter while the other is split down themiddle. This split may results in a blurring of the information in twoof the primaries. In one embodiment, by assuming the input pixels areoffset ¼ pixel to the left, the red and green sub-pixels becomeperfectly aligned while the white and blue sub-pixels use the splitfilter. In another embodiment, it may be possible to align the pixelswith the highest luminosity, so if the input pixels are assumed to beoffset ¼ pixel to the right then the white and blue sub-pixels arealigned while the red and green sub-pixels are split across an inputpixel. The assignment of the above filters would be modified for a panelbased on the arrangement of FIG. 5B, as would be obvious from thisteaching to one skilled in the art.

This split may be further processed by using a cubic filter to move thephase of the input data for the split sub-pixels until they are alsocentered. This may be accomplished by using the following cubic filterto do this ½ pixel offset:

One-Half (½) input pixel cubic offset filter −16 144 144 −16

This offset filter may be easy to implement as shifts and adds inhardware or software. The input pixels are assumed to be shifted ¼ pixelone direction for half of the output sub-pixels and they may be renderedwith the diamond filter. The other 4 sub pixels may have their inputshifted with the above cubic filter then they may also be rendered withthe diamond filter.

In hardware, it is easy to implement the above cubic shift on the inputdata as it flows through the SPR controller. In software, it is oftenmore convenient to convolve the cubic filter with the diamond filter andperform a single filtering operation on the input for the non-alignedsub-pixels. In this case, the following combined filter kernel is used:

0 −2 18 18 −2 0 −2 10 88 88 10 −2 0 −2 18 18 −2 0

For the cases when the sub-pixels are aligned or brought into alignmentwith cubic filters, the standard cross-color or cross-luminositysharpening filter may be used. If, however, the input pixels remaincentered around pairs of output sub-pixels, then it is possible to usethe following cross-luminosity filters for sharpening:

−28 0 −4 −4 0 −28 0 72 0 0 72 0 −28 0 −4 −4 0 −28

FIG. 7 is yet another embodiment of the novel high brightness layoutsmade in accordance with the principles of the present invention. It maybe seen that the red and green—as well as the blue and white—subpixelsare created on a checkerboard pattern. It will be appreciated that thesimilar filters as described above may be used on this alternative,although they may be used in a different order or slightly differentfilter kernels than the other layouts.

FIG. 8 is yet another embodiment of a high brightness color filterarrangement as made in accordance with the principles of the presentinvention. In FIG. 8, the subpixels are shown (in grid 802) having itscolored subpixels with a 2:3 aspect ratio but white sub-pixels with anaspect ratio of 1:3. In this embodiment, arranging three rows of threecolor pixels in a mosaic or diagonal stripe arrangement, the layoutbecomes color balanced. It should be noted that, with a narrow whitesubpixel next to each color sub-pixel, each logical pixel has a brightluminosity center. In one embodiment, the input pixels may be centeredon these white sub-pixels, so the white value may be simply sampled ateach input location. All the color sub-pixels may be split in thisalignment, but due to the diagonal stripe layout, the area resamplingfilter may be a tilted hexagon as in FIG. 9.

Looking at FIG. 9, input image data grid 900 is shown. Superimposed ongrid 900 is target subpixel grid 802. Centers of red subpixels and theirassociated resample areas (centered around dots 902 a, 902 b, and 902 c)are also shown. In one embodiment, the hexagonal resample areas may becalculated by considering the surrounding red subpixel centers anddrawing even boundaries lines between the centers. For example, redcenter 902 a and its associated resample area has a boundary line 906which substantially bisects the line between center 902 and red center904. Similarly, lines 908 and 910 substantially bisect the lines betweencenter 902 a and 902 b and 902 c respectively. It will be appreciatedthat other resample area shapes may be formed in other manners for otherembodiments. It suffices that the resample areas are substantiallycorrelated with input image data in a spatial manner. It will also beappreciated that the green color plane—or any other color plane—may betreated similarly.

The resulting filter kernels may be identical for every sub-pixel ofevery color and could be a 4×3 filter. However, when converted to 8bitintegers, the small areas on the right and left became very small andmay be discarded, resulting in the following exemplary filter:

$\begin{matrix}40 & 12 \\76 & 76 \\12 & 40\end{matrix}$

Alternatively, the ½ pixel cubic offset filter may be used to adjust thephase of the input pixels until the pseudo-samples land on the centersof the output sub-pixels again. In this case, the area resample filtersmay become a 3×3 filters, as given below. Once centered like this, it ispossible to use a cross-luminosity sharpening filter for this alignment,as given below.

Area Resampling Cross-Luminance Sharpening 16 35 0 −16 0 0 35 84 35 0 0−35 0 35 16 0 102 0 −35 0 0 0 0 −16

As with the other layouts disclosed herein, the cubic interpolationaccomplishing the ½ pixel alignment may be done on a scan-line basis andmay be done to the input data as it arrives. However, in softwareimplementations, it may be convenient to convolve the cubic filter withthe above two filters to do each sample in a single step. In this case,the combined cubic and area resampling filter is given below on the leftwith the combined cubic and sharpening filter on the right:

−1 7 29 19 −2 0 1 −9 −9 1 0 0 −2 14 64 64 14 −2 0 0 2 −20 −20 2 0 −2 1929 7 −1 0 −6 58 58 −6 0 2 −20 −20 2 0 0 0 0 1 −9 −9 1

In another embodiment, the layout of FIG. 8 may use a cubic arearesampling filter above, but may use a non-cubic cross-luminosityfilter. This filter may be desirable for images with sharp edges such astext.

Sub-Pixel Rendering Filters and Offset Assumptions

Apart from use on high brightness layouts, the techniques of performingimage data offsets to achieve advantageous filter kernels is alsoapplicable to the full range of other subpixel layouts (e.g. 3-color,4-color, 5-color, etc.) disclosed herein and in the applicationsincorporated by reference. The technique of area resampling may bethought, in one sense, in a geometric model for calculating the filterkernels. A picture of a target layout may be typically drawn on top of agrid of source RGB pixels. A center point, called a resample point, maybe chosen for each of the sub-pixels in the target layout. Shapes,called resample areas, may be drawn which enclose substantially all ofthe area that lies closer to one resample point than any other resamplepoint of the same color.

FIG. 10 depicts a three-color subpixel repeating pattern 1000 thatsubstantially comprises red 1002, green 1004 and blue 1006 subpixels ofapproximately the same size. Grid lines 1008 depict an overlay of sourceinput image data that should be remapped to the target subpixel layout.As may be seen, the input image data grid seems to split the bluesubpixels in some ratio (e.g. one half). In the case of the layout ofFIG. 10, these blue resample areas are simple rectangles. The resamplepoints for red and green were chosen to make the resample areas turn outto be diamonds, or squares rotated 45 degrees as shown in FIG. 11. Inboth squares and diamonds, the shapes of the resample areas were simpleenough that the intersection of the areas of the source pixels and theresample areas could be calculated analytically or geometrically.

These choices for red and green resample points are in some sense asimplification, done to make the resample areas easier to calculate andthe resulting filters less expensive to implement in hardware. In thesefilter designs, the resample points of the red and green sub pixels werenot placed at the centers of the sub-pixels, but were moved slightlyleft or right to make them align with the centers of the source pixelsor logical pixels, as seen in FIG. 11. If these resample points areplaced substantially at the center of each target sub-pixel, then theresample areas become more complicated asymmetrical diamond-like shapes,as seen FIG. 12. These shapes sometimes resemble kites flyingsideways—so the resulting filters are termed “kite filters”. These newshapes may be more difficult to calculate geometrically and they maychange with every variation of any given subpixel layout. In some cases,it may be advantageous to leave the resample points substantially on thecenter of the subpixels. For example, this may reduce color error insome images. In other cases, it may be advantageous to move the resamplepoints substantially to the center of the resample area. For example,this may simplify the filters and make implementing them in hardwareless expensive.

New Filter Generation:

One embodiment of generating resample areas and their filter kernelswill now be described:

-   (1) A first step is to accept a list of resample points and create a    picture or other representation in a bitmap file.-   (2) Each pixel in this image is substantially compared against all    the resample points to find out which resample point is closest. In    doing this, it may be desirable to consider all neighboring resample    points above, below, left, right as well as in all four diagonal    direction.-   (3) A second pass through the bitmap image may be taken and the    count of the number of pixels that are tagged as closest to one    resample point may be an approximation of the resample area for that    resample point. The number of tagged pixels inside each source pixel    square may also be counted.-   (4) The ratio of these two numbers may be an approximation of the    coefficient for the filter kernel for each source pixel. The bitmap    image can be displayed or printed out to see what the resulting    shape looks like and to verify that the shapes make sense.

It will be appreciated that other methods and steps may be taken togenerate filter kernels for the mapping of input image data to a targetsubpixel layouts. It suffices for the purposes of the present inventionthat the filter kernels extract out image data that is substantiallycorrelated to the target subpixels in a spatial manner.

Translating Edge Assumptions:

Using a point near the exact center of the sub-pixel as the resamplepoint, in some cases, may be simplified by changing the edgeassumptions. A simplifying assumption of placing a target layout (suchas shown in the '225 application and other applications incorporatedherein) on top of 4 source pixels may result in diamonds and boxes thatmay be out-of-phase with the input pixels. One example is seen in FIGS.13A, 13B, and 13C—depicting the red, green and blue resample areasrespectively. Translating all the resample points together is not asimplification since the choice of edge alignment could be arbitrary. Inmany of the layouts, a slight shift to the left of all the resamplepoints resulted in much simpler filters and sharper greens. For example,such suitable shifts result in the resampling areas seen in FIGS. 14A,14B, and 14C.

Adjusting Center Locations:

When points close to the exact sub-pixel center are used as the resamplepoints for the layout shown in FIG. 15 (e.g. two blue subpixels 110staggered within a substantially checkerboard pattern of red and greensubpixels), a large set of different filters may result. For example,FIGS. 16A, 16B and 16C are one possible set of filters for such alayout.

In another embodiment, both the red resample points can be movedslightly to make the red filter areas diamonds, as may be seen in FIG.17A—with FIGS. 17B and 17C depicting the green and blue filtersrespectively. Yet another embodiment might be a combination oftranslation and adjustments to make the two green areas thediamonds—while the red and blue would remain “kites”, as is shown inFIGS. 18A, 18B and 18C. This may have the effect of keeping greensharper. Since green has most of the luminosity, this may result in asharper total image. In addition, having all the green resample pointscentered on input pixels would allow them to be sharpened withcross-color sharpening.

Decimation Filters:

Adjusting the relationship between source pixels and the subpixels inthe layout shown in FIG. 15 might also help with decimating RGB datainto such a display. As may be seen in FIG. 15, there may be a red or agreen sub-pixel completely inside each source pixel. In a simple-toimplement hardware decimation mode, the correct red or green primaryvalue from the underlying RGB pixel could be copied directly into thetarget sub pixels. The blue sub-pixels may be split and may be averagedor even have one of the two source blue values used arbitrarily withoutnoticeable problems in the image.

If the edges of the source pixels are aligned with the target layout,one of the green sub pixels may be split between two source pixels.Averaging the two source greens may produce a fuzzy image; while pickingone source value may result in some degradation of image quality.Alternatively, the remapping grid 1502 could be shifted between thesource pixels so that the green sub pixels are not split, as may be seenin FIG. 15. This will result in one of the red sub pixels being splitbetween two source pixels, but since green contributes more to theluminosity of the image, splitting one of the reds may not degrade theimage as much.

While the present disclosure of invention has been provided withreference to exemplary embodiment herein, it will be understood by thoseskilled in the art and in view of the foregoing that various changes maybe made and equivalents may be substituted for elements thereof withoutdeparting from the scope of the present teachings. In addition, manymodifications may be made to adapt a particular situation or material tothe teachings without departing from the essence thereof. Therefore, itis intended that the present teachings not be limited to the particularembodiments disclosed, but rather that the present disclosure ofinvention will include all embodiments falling within the spirit andscope of the teachings provided herein.

1. A method for rendering a colors- and/or grays-filled imagerepresented by a received input image data signal onto a display panel;said display panel having a display area substantially populated bytessellation of a subpixel repeating group comprised of at least onewhite subpixel that is configured to produce a white light and of aplurality of different primary color subpixels configured torespectively produce respective non-white lights of respective primarycolors; the method comprising: receiving an input image data signalrepresenting a corresponding colors- and/or grays-filled image forrendering on said display panel, the corresponding input image beingrepresented by the input image data signal as a grid of input pixels,each input pixel having a respective mix of input primary colors and/oran input white component and a respective input luminance; defining arespective primary color plane for each different color respectivelyproducible by said plurality of different primary color subpixels, therespective color plane being populated by respective colored lightsource points corresponding to centers in the display area of thesubpixels configured to respectively produce the primary color of therespective primary color plane; defining a white color plane, said whitecolor plane being populated by respective white light source pointscorresponding to centers in the display area of the subpixels configuredto produce said white color; for each individual primary color planerespectively defined by respective ones of the same-colored primarycolor subpixels of the display area, using a subpixel rendering processfor subpixel rendering corresponding colored pixel components of saidinput image data signal so as to thereby produce subpixel rendered imagedata representing how the primary color subpixels are initially plannedto be driven so as to reproduce the colored input pixel components ofthe input image; for the white color plane, using the subpixel renderingprocess for mapping corresponding white input pixel components of saidinput image data signal onto the white color plane so as to therebyproduce subpixel rendered image data representing how the whitesubpixels are initially planned to be driven so as to reproduce thewhite input pixel components of the input color image; defining aluminosity plane, the luminosity plane having a grid corresponding tothat of the grid of input pixels represented by the input image datasignal, and populating the luminosity plane with luminosity valuesderived from the corresponding input pixels represented by the inputimage data signal; using the populated luminosity plane to define aluminance signal; and sharpening the subpixel rendered image dataproduced by said subpixel rendering process with the defined luminancesignal.
 2. The method of claim 1 wherein a format of said input imagedata is one member of a group of different formats, said group ofdifferent formats including at least: RGB, sRGB, and YCbCr.
 3. Themethod of claim 1 wherein said plurality of primary colored subpixelscomprises subpixels in at least three primary colors; and wherein saidstep of subpixel rendering comprises subpixel rendering said input imagedata for each of at least three color planes.
 4. The method of claim 1wherein the step of sharpening said subpixel rendered image data furthercomprises sharpening at least one color plane with luminance data. 5.The method of claim 4 wherein the step of sharpening at least one colorplane with luminance data further comprises sharpening with a differenceof Gaussian filter.